Cooperation in spatial Prisoner's Dilemma with two types of players 
for increasing number of neighbors 
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We study a spatial two-strategy (cooperation and defection) Prisoner's Dilemma game with two types (A 
and B) of players located on the sites of a square lattice. The evolution of strategy distribution is governed 
by iterated strategy adoption from a randomly selected neighbor with a probability depending on the payoff 
difference and also on the type of the neighbor. The strategy adoption probability is reduced by a pre-factor 
(w < 1) from the players of type B. We consider the competition between two opposite effects when increasing 
the number of neighbors (k = 4, 8, and 24). Within a range of the portion of influential players (type A) 
the inhomogeneous activity in strategy transfer yields a relevant increase (dependent on w) in the density of 
cooperators. The noise-dependence of this phenomenon is also discussed by evaluating phase diagrams. 

PACS numbers: 89.65.-.S, 89.75.Fb, 87.23.Cc, 05.50.+q 



The investigation of the spatial evolutionary Prisoner's 
Dilemma (PD) games expands progressively since Nowak and 
May |1] reported the maintenance of cooperative behavior 
among selfish players. In these models the PD game |2] rep- 
resents a pair interaction between two players who can ei- 
ther cooperate (C) or defect (D) and their income depends 
on both choices in a way forcing both rational (selfish) play- 
ers to choose D while they would share equally the maximum 
total payoff for mutual cooperation. 

In the first model the players are located on a square lat- 
tice, they can follow one of the two pure strategies (D and C) 
and their income comes from PD games with the neighbors. 
During a synchronized strategy update the players adopt the 
strategy from the neighbor receiving the highest score. After 
this pioneering work many modified versions of the original 
model have been suggested and studied (for recent surveys see 
H, Let us mention only a couple of exarnples: in some 
models larger set of strategies was used IHI^Q], in the evo- 
lutionary rules noises |8, 9, IQj were introduced that can help 
the cooperative behavior illl[l2,IT3ll . and the spatial structure 
was also extended by locating the players on different graphs 
giving a better description about the connections in human so- 
cieties ifTilfTslll^fTTTl . In the last years the concept of interac- 
tion and learning graphs are distinguished lfT8l[T9ll20ll and the 
research of the co-evolution of strategy distribution and these 
graph has also become a promising topic ll2ll l22l |23[]. It is 
found, furthermore, that different types of personahty 
and inhomogeneous activity in the strategy adoption can also 
support cooperation iflSl uoL uA |28'] particularly if some dis- 
tinguished players have higher influence to spread their strate- 
gies Mm- 

In the latter case the relevant increase in the frequency of 
cooperators is related to a phenomenon described previously 
by Santos et al. ll3ll[32ll who studied evolutionary PD games 
on scale-free networks with an evolutionary rule exploiting 
the high income for players (called hubs) who have a large 
number of neighbors. As a result, on the scale-free networks 
the strategy of hubs become an example to be followed by 
their neighborhood. Thus, the hubs as influential players face 
the consequence of the imitation of their own strategy that 
increases (decreases) the income of cooperative (defective) 



hubs. After a short transient process this phenomenon favors 
the spreading of cooperation because the influential players 
can also adopt strategy from each other for suitable connectiv- 
ity structures. Evidently, in the absence of links between in- 
fluential players the mentioned mechanism cannot help coop- 
erators to beat defectors [ 3ll [12, [Ht] . Recent studies I Sim 
have indicated that the presence of linked influential players 
on scale-free graphs can efficiently promote cooperation (even 
for normalized payoffs) if the capability of strategy spreading 
differs from player to player These results raise many inter- 
esting questions about the impact of the size of neighborhood 
on the frequency of cooperators for inhomogeneous activity 
in the strategy transfer. 

In the present work we study the competition between two 
opposite effects emerging if the average number of neighbors 
is increased. On one hand, the above described mechanism 
(supporting the spreading of cooperation for inhomogeneous 
strategy transfer capability) is enhanced when choosing larger 
and larger k. On the other hand, the increase of the number of 
neig hbors, k, is beneficial for defectors on regular networks 
||34[3l[36i[37t[38i[39tl40|l. Here it is worth mentioning that 
the mean-field approximation (predicting the extinction of co- 
operators in the evolutionary PD games [3, 4]) gives a simple 
explanation of this phenomenon. The scope of the present pa- 
per is to explore the impact of these two opposite effects by 
comparing results obtained for three different sizes of neigh- 
borhood. More precisely, the studied types of neighborhood 
are the von Neumann neighborhood including only the nearest 
neighbors (fc — 4), the Moore neighborhood with nearest and 
next-nearest neighbors {k ~ 8), and the case of = 24 where 
players within a 5 x 5 box of sites are neighbors of the central 
player (self-interaction is excluded). Monte Carlo (MC) sim- 
ulations are used to study systematically the effects of payoff, 
number of neighbors, and inhomogeneous capability of strat- 
egy transfer (for a fixed noise level) on the average number of 
cooperators. 

For these evolutionary PD game models two types of play- 
ers (rix — A or E) are located on the sites x of a square lattice 
with a concentration of v and (1 — i^) and their random initial 
distribution remains unchanged (quenched) during the simula- 
tions. The income of player x comes from one-shot PD games 
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with her neighbors, that is. 



(1) 



where the sum runs over all neighboring sites (fi^.) of player 
X, the payoff matrix is defined as suggested by Nowak and 
May il. 



A= ( [j J ) , 1<5<2, 



(2) 



and the defective and cooperative strategies are denoted by 
unit vectors as 



= D = 



(3) 



The evolution of strategy distribution is governed by random 
sequential strategy update representing strategy adoption from 
a randomly chosen site y to one of its neighbors x with a prob- 
ability 



1 + exp[(P, - Py)/kK] 



(4) 



dependent on the difference of normahzed payoffs (e.g. [Px^ 
Py)/k) for later convenience of comparisons. For this strat- 
egy adoption probability the meaning of the parameter K is 
analogous to the temperature as introduced in the kinetic Ising 
model and characterizes the magnitude of payoff noise affect- 
ing the decision of player x |9, 10]. The multiplicative factor 
Wy denotes the strategy transfer capability of player y, 



if Hy = A 



if n„ 



B 



< w < 1 



(5) 



In this notation players of type A represent those individuals 
who can easily convince their neighbors to adopt the strategy 
they are just following. This personal feature can be related to 
age, reputation, etc. 

For all the three cases studied here the simulations are per- 
formed on an i X L square of sites with periodic boundary 
conditions. The evolution of the spatial distribution of the C 
and D strategies starts from an uncorrelated initial state where 
cooperators and defectors are present with the same probabil- 
ity. When repeating the above described elementary steps the 
system develops into a final stationary state characterized by 
the average density of cooperators {p). After a suitable re- 
laxation time tr p is determined by averaging the density of 
cooperators over a time ta- Typical (maximum) value of pa- 
rameters used in our simulations are the following: L = 400 
(1600); U ~ ta ^ 10-* (10*^) MCS [during one MC step 
(MCS) each player has a chance once on average to adopt one 
of the neighboring strategies]. Pronounced long relaxations 
were observed at the large noise limit. 

Before discussing the behaviors of the above systems we 
briefly recall some general features of the homogeneous sys- 
tem {v — Q or 1) IHTIl . The average (total) payoff increases 
monotonously with p independent of the initial strategy distri- 
bution. Furthermore, in each homogeneous system the value 



of p decreases monotonously from 1 to if 6 is increased 
within a region of b (6^?(is:) < b < b^^^{K)) where the 
strategies C and D coexist. For all the three types of neigh- 
borhoods b'^^^ (K) (b^,^ {K)) tends to 1 from below (above) if 
if ^ cxo. In other words, in the strong noise limit {K oo) 
the systems reproduce the behavior of the mean-field model, 
that is p drops suddenly from 1 to at 6 = 1. In the oppo- 
site case {K — > 0) the limit values of b'"^^ and b''^ depend 
on k. When decreasing K the upper critical value of b tends 
monotonously to a value (6^^'' (0) larger than 1 if fc = 8 or 24. 
On the contrary, for k = A, the function b''^2 i^) has a local 
maximum at K = K, 



opt 



0.08 (b'-^\K 



Kopt) ^ 1.08) 



and approaches 1 if if ^ 0. 

In the light of the above results we first study the density 
of cooperators (p) when varying the portion of players of type 
A for fixed values of payoff (b), strategy transfer capability 
(w), and noise (K). The latter was chosen to present optimal 
cooperation for k — A system (i.e., K = Kopt)- The MC data 
are compared in Fig. [T] for the three types of neighborhood. 
For the sake of comparison, we selected such a high value of b, 




FIG. 1: Density of cooperators as a function of the portion of A 
players if 6 = 1.05, K — 0.1 and w — 0.1 for three different 
neighborhoods: fc = 4 (open squares); 8 (closed squares); and 24 
(closed triangles). 

which prevents cooperation in the homogeneous model of fc = 
24. Figure [U shows that the highest density of cooperators can 
be observed (at ~ or 1) for k = 8 where the overlapping 
triangles in the connectivity structure support the spreading 
(maintenance) of cooperation as discussed in lfT2ll . The further 

(k) 

increase of fc, however, yields a decrease in both p and b)^2 
il^l tending to the behavior of mean-field model. This is the 
reason why cooperators become extinct in the final stationary 
state for the homogeneous system at fc = 24. 

Figure [T] demonstrates clearly the existence of an optimum 
composition (defined by the maximum in p) of the players 
A and B. The presence of distinguished players results in a 
relatively higher impact on cooperation level for larger k. In 
agreement with the expectations, the more neighbors the play- 
ers have, the smaller portion of influential players (type A) 
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are capable to achieve the highest increase in p. The resuhant 
asymmetry can be observed in the function p{v) for k — 24. 
For the largest neighborhood our simulations have clearly in- 
dicated that cooperators can remain alive only within a region 
of V with boundaries dependent on w and K. It is expected, 
that this region, vi{w,K) < v < V2{w,K), shrinks if we 
increase k further. 

As the largest effect is found for the largest neighborhood, 
henceforth our attention will be focused on the system of 
k = 24. Figure |2] illustrates the increase of the density of 
cooperators when varying the composition of players A and 
B for several values of w at a fixed payoff and noise level. 
When the difference is small, typically when 1/w < 2, the 
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FIG. 2: Density of cooperators vs. v for five different values of the 
reduced strategy transfer capability (w = 0.01, 0.02, 0.05, 0.1, and 
0.2 from top to bottom) at 6 = 1.05, k = 24, and K = 0.1. 

cooperators cannot remain alive at the given payoffs and noise 
independently of the actual composition of A and B play- 
ers. If the ratio l/w is increased then the cooperators can 
survive within the above mentioned region of v. This inter- 
val becomes wider and wider while the maximum value of 
p increases monotonously until reaching its saturation value 
ip = 1). Consequently, we can observe four subsequent tran- 
sitions in Fig. |2]if increased for sufficiently high values of 
the ratio 1/w. Apparently the density of cooperators tends 
to a hmit profile if 1/w — * oo. We have to emphasize that 
the rigorous analysis of the asymptotic behavior becomes dif- 
ficult because the transient time increases with the ratio 1/w 
particularly at small values of v. 

We have also studied the effect of the variation of w on the 
cooperation level at different payoffs (&). To avoid additional 
effects the noise level is fixed at a composition (v = 0.2) close 
to its optimum value. The results, summarized in Fig.|3] illus- 
trates that the curves p{b) shift to larger b values if the ratio 
1 /w is increased. (For comparison, the left curve shows the 
results obtained in the homogeneous system). The plotted re- 
sults refer to a shift proportional to ln(l/u'). Due to the above 
mentioned increasing run time if we choose larger values of 
1 /w, the more rigorous (numerical) confirmation of this trend 
goes beyond the scope of the present work. Instead of it we 
have concentrated on the effect of noise for the two extreme 




FIG. 3: Density of cooperators as a function of b for different values 
of 1/w (w = 1, 0.2, 0.05, 0.02m and 0.005 from left to right) at fixed 
noise level (K = 0.1), composition (i/ — 0.2), and neighborhood 
(k = 24). 

neighborhoods (fc = 4 and 24) at a fixed portion of players A 
and B. For this purpose we have performed systematic MC 
simulations to determine the critical values fe^i and 5c2 for a 
fixed ratio of strategy transfer capability (1/w — 50). 
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FIG. 4: The upper and lower critical values of 6 for A: = 24 and 
v = 0.2 (open squares connected with dashed lines). Results for 
k = 4 and i/ — 0.5 are denoted by closed squares (connected with 
solid lines) ai w — 0.02. The dotted line illustrates the prediction 
of mean-field approximation in the homogeneous system (bj;™'^ = 
^c™'' ~ 1 for arbitrary K). 

Figure |4] can be interpreted as a phase diagram where the 
connected data represent phase boundaries. Between the up- 
per and lower critical points strategies C and D coexist. 
Above (below) this region only defectors (cooperators) remain 
alive in the final stationary states. For both cases the system 
behavior is not affected by the spatial inhomogeneities in the 
low noise limit, in agreement with the previous results ll29tl . 
In other words, the relevant improvement in the maintenance 
of cooperation appears in the noisy systems even for the limit 
K oo. In contrary to the prediction of mean-field theory 
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the present data indicate clearly that cooperators and defectors 
can coexist within a region of b if K goes to infinity, that is, 
bci and bc2 tends to two distinct limit values for both types of 
neighborhood. This latter feature has already been confirmed 
qualitatively by the pair approximation for k — 4 [29]. No- 
tice, furthermore, that the larger neighborhood yields a larger 
increase in the value of bd and bc2 when applying optimum 
composition of players A and B for both systems. 

In summary, within the framework of evolutionary PD 
games, the present investigation of the effect of the inhomoge- 
neous strategy transfer capability on the cooperative behavior 
has indicated a relevant increase in the density of cooperators 
if the fraction of influential players was close to the optimum 
value dependent on the number of neighbors (range of inter- 
action) if two types of strategy transfer capability (represented 
by the players A and B) are distinguished. It is found that the 
larger neighborhood with a less fraction of influential players 
(type A) can be more beneficial for the whole system due to 
the imitation mechanism rewarding (punishing) cooperation 
(defection) for the influential players. The improvement of 
cooperation increases with the ratio of strategy transfer capa- 



bility (l/w) between players of type A and B. Furthermore, 
the maintenance of cooperation is supported remarkably by 
the mentioned effect in the high noise limit where the region 
of coexistence is broadened and shifted to higher values of b. 
For small fraction of players A one can think that the com- 
petition between the influential players surrounded by their 
followers can be characterized by an effective (rescaled) pay- 
off matrix favoring cooperation (as it appears on evolving net- 
works |21]) while their competitions are disturbed by those 
players of type B who do not belong to the neighborhood of 
any influential player. These latter B players can mediate an 
interaction between the influential players and/or preserve the 
defective behavior. Further research is requested to clarify the 
relevance of these opposite effects. 
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